Submitted to the IEEE Transactions on Information Theory, Oct. 2005 



Vector Gaussian Multiple Description with 
Individual and Central Receivers 1 

Hua Wang and Pramod Viswanath 2 
Abstract 

L multiple descriptions of a vector Gaussian source for individual and central re- 
ceivers are investigated. The sum rate of the descriptions with covariance distortion 
measure constraints, in a positive semidefinite ordering, is exactly characterized. 
For two descriptions, the entire rate region is characterized. Jointly Gaussian de- 
scriptions are optimal in achieving the limiting rates. The key component of the 
solution is a novel information-theoretic inequality that is used to lower bound the 
Q ' achievable multiple description rates. 

in . 

1 Introduction 



In the multiple description problem, an information source is encoded into L packets 
and these packets are sent through parallel communication channels. There are several 
receivers, each of which can receive a subset of the packets and needs to reconstruct 
the information source based on the received packets. In the most general case, there 
are 2 L — 1 receivers and the packets received in each receiver correspond to one of 2 L — 
1 subsets of {1, . . . , L}. A long standing open problem in the literature [1-10] is 
to characterize the information-theoretic rate region subject to the specified distortion 
constraints. Practical multiple description codes have been discussed in [11-18] and 
recent work [19, 20] has considered the multiple description problem in the context of 
the distributed source coding scenario. Optimal descriptions of even the Gaussian source 
with quadratic distortion measures have not been fully characterized. In the special 
case of two descriptions of a scalar Gaussian source with quadratic distortion measures, 
however, the entire rate region has been characterized in [1]. 

x 

Our focus is on L descriptions of a memoryless vector Gaussian source forwhere L 
individual and a single common receiver (cf. Figure^). Each receiver needs to reconstruct 
the original source such that the empirical covariance matrix of the difference is less than, 
in the sense of a positive semidefinite ordering, a "distortion" matrix. In this setting, the 
symmetric rate multiple description problem of a scalar Gaussian source with symmetric 
distortion constraints has been characterized in [7,8,10], but a complete understanding 
of all other rate-distortion settings is open. 

1 This research was sponsored in part by NSF CCR-0325924 and a Vodafone US Foundation Fellow- 
ship. 

2 The authors are with the Department of Electrical and Computer Engineering and the Coor- 
dinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana IL 61801; e-mail: 
{huawang , pramodv}@uiuc . edu 
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Figure 1: MD problem with only individual reconstructions and central reconstruction 



Our main result is an exact characterization of the sum rate for any specified L + 1 
distortion matrix constraints. With L = 2, we characterize the entire rate region. Our 
contribution is two fold: 

• First, we derive a novel information-theoretic inequality that provides a lower bound 
to the sum of the description rates. The key step is to avoid using the entropy 
power inequality, which was a central part of the proof of two descriptions of the 
scalar Gaussian source in [1]: the vector entropy power inequality is tight only 
with a certain covariance alignment condition, which arbitrary distortion matrix 
requirements do not necessarily allow. 

• Second, we show that jointly Gaussian descriptions actually achieve the lower bound 
not by resorting to a direct calculation and comparison, which appears to be difficult 
for L > 2, but instead by arguing the equivalence of certain optimization problems. 

Consider another two description problem of a pair of jointly Gaussian memoryless 
sources as depicted in Figure El There are two encoders that describe this source to 
three receivers: receiver i gets the description of encoder i, with i — 1,2 and the third 
receiver receives both the descriptions. Suppose receiver i is interested in reconstructing 
the ith marginal of the jointly Gaussian source, with i — 1,2. The third receiver is 
interested in reconstructing the entire vector source. This description problem is closely 
related to the vector Gaussian description problem that is the main focus of this paper. 
We exploit this connection and characterize the rate region where the reconstructions 
have a constraint on the covariance of error at each of the receivers (in the sense of a 
positive semidefinite order). 

We have organized the results in this paper as follows. In Section |2] we give a formal 
description of the problem and summarize our main result. The derivation of a lower 
bound is in Section El In Section |U we provide an upper bound and provide conditions for 
the achievable sum rate to meet the lower bound. We see in Section El that the conditions 
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Figure 2: Multiple Descriptions with separate distortion constraints. 



are indeed satisfied in the special case of a scalar Gaussian source. The solution in the 
case of the more complicated vector Gaussian source is in Section The solution to the 
multiple description problem depicted in Figure|2]is the topic of Section l7!Tl Finally, while 
the characterization of the rate region of general multiple descriptions of the Gaussian 
source (with each receiver having access to some subset of the descriptions) is still open, 
we can use the insights derived via our sum rate characterization to solve this problem 
for a nontrivial set of covariance distortion constraints; this is done in Section [7"21 

A note about the notation in this paper: we use lower case letters for scalars, lower 
case and bold face for vectors, upper case and bold face for matrices. The superscript t 
denotes matrix transpose. We use I and to denote the identity matrix and the all zero 
matrix respectively, and diag{pi, . . . ,p n } to denote a diagonal matrix with the diagonal 
entries equal topi, . . . ,p n . The partial order y (y) denotes positive definite (semidefinite) 
ordering: A y B (A y B) means that A — B is a positive definite (semidefinite) matrix. 
We write N{jJ,, Q) to denote a Gaussian random vector with mean \x and covariance Q. 
All logarithms in this paper are to the natural base. 



2 Problem Setting and Main Results 



2.1 Problem Setting 



The information source {x[m]} is an i.i.d. random process with the marginal distribution 
A/"(0,K x ), i.e., a collection of i.i.d. Gaussian random vectors. Denoting the dimension 
of {x[m]} by N, we suppose that K. x is an N x iV positive definite matrix. There are 
L encoding functions at the source, encoder I encodes a source sequence, of length n, 



x n = (x[l], . . . , x[n])* to a source code C\ n ' = f t 
is sent through Zth communication channel at the rate Ri 
individual receivers and one central receiver. 



, for I = 1 . . . L. This code c[ n) 



k log l^| 



There are L 
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For 1 = 1, ... L, the Ith individual receiver uses its information (the output of the /th 

channel) to generate an estimate x™ = gf^ ^/^(x n )^ of the source sequence x™. The 

central receiver uses the output of all the L channels to generate an estimate Xq of the 
source sequence x n . Since we are interested in covariance constraints, the decoder maps 
can be restricted to be the minimal mean square error (MMSE) estimate of the source 
sequence based on the received codewords. So, 



E 



x"|// n) (x") 



1 = 1, 



Xq = E 



X 



Suppose the reconstructed sequences satisfy the covariance constraints 
1 " 



(1) 



— E |~(x[m] — x/[m])*(x[m] — x/[m]) 
n l 

m=l 
1 n 

— E (x[m] — x [m])*(x[m] — x [m]) 



^ D ; , 1 = 1, L, 
4 D , 



(2) 



m=l 



then we say that multiple descriptions with distortion constraints (D 1? . . . , D^, D ) are 
achievable at the rate tuple (R±, . . . , Rl). 

The closure of the set of all achievable rate tuples is called the rate region and is denoted 
by 1Z*(K X , Di, . . . , D £ , D ). Throughout this paper, we suppose that -< D -< -< 
K,. V/ 1 I.:' 



2.2 Sum Rate 

Our main result is the precise characterization of the sum rate of multiple descriptions 
for individual and central receivers. 

Theorem 1. For distortion constraints (D 1; . . . , D L , D ), the sum rate is 



sup \ log 

K z >-0 ^ 



\ 



IK^IK^ + K.I^IDq + K, 

|D | EI |A + K,| 

i=i 



(3) 



3 That D ^ D ; , is without loss of generality is seen by applying the data processing inequality for 
mmse estimation errors; having more access to information can only reduce the covariance of the error in 
a positive semidefinite sense. Similarly, =<; Do is also not interesting; here we simplify this condition 
and take Do -< K x . 
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This sum rate is achieved by a jointly Gaussian random multiple description scheme: 
let wi, • • • , Wi be zero mean jointly Gaussian random vectors independent of x, with 
the positive definite covariance matricex (wj, • • • , w^) denoted by K. w . Defining 

U; = X + Wl, I = 1, L, 

we consider such that 



Cov[x|ui] d =E[(x - E[x|u,])*(x - E[x|uj]) 
Cov[x|ui, u L ] = f E (x-E[x|ui, Ui])*(x - E[x|ui, u L ]) 



4 D/, 1 = 1, L, 

^D . 



(4) 



To construct the code book for the Ith description, draw e nRl uf vectors randomly ac- 
cording to the marginal of u^. The encoders observe the source sequence x n , look for 
codewords (u™ , . . . , u£) that are jointly typical with x n and send the index of the re- 
sulting u" through the /th channel, respectively. The /th individual receiver uses this 
index and generates a reproduction sequence E[x n |u[ l ] for I = 1 . . . L, the central receiver 
uses all the L indices to generate a reproduction sequence E[x n |u™, . . . , u£]. For every 
K w satisfying (jlj), the rate tuple (R%, . . . , Rl) satisfying 

Y\\K x + K Wl \ 

J2 R i>Y, h W- h ( u i> leS W = o l °Z leS \k I > V5C{1, L} (5) 



les leS 



I K-^g 



is achievable by using this coding scheme, where K ws is the covariance matrix for all 
wi, I G S, and K W; = E[w*W;]. In particular, the achievable sum rate is 



L 

n i k z + 
1 1 i=i 



9 l0g \tT~\ ■ ^ 

& \""W I 

We denote this ensemble of descriptions, throughout this paper, as the jointly Gaussian 
description scheme and the time sharing between them as the jointly Gaussian description 
strategy. We show that jointly Gaussian description schemes are optimal in achieving 
the sum rate ©• 



2.3 Rate Region for Two Description Problem 

For two descriptions, we can characterize the entire rate region. 

Theorem 2. Given distortion constraints (D\, D2, Do), the rate region for the two 
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Figure 3: Rate region for two description problem 



description problem for an i.i.d. A/"(0,K x ) vector Gaussian source is 



K*(K X , D 1; D 2 , D ) = ^ 



Ri > - log 

2 5 D, 



I = 1, 2 



i?l + i? 2 > SUp - lo£ 
K z ^0 2 



IK*! 


IK. + K,! 


Po + K„| 


|Do| 


p! + K,| 


p 2 + K 2 | 



(7) 



We show that if the distortion constraints (D 1; D 2 , D ) satisfy Dq + K^ — D^ — D 2 y 
and Dq 1 + K^ 1 — D^ 1 — D^ 1 y 0, we can get the optimizing K 2 by solving a matrix 
Riccati equation. An illustration of the rate region is shown in Figure El In this case, if 
we let K, 



[D[ 1 — Kj, 1 ] 1 for Z = 0, 1, 2, then the optimizing K 2 is 



where 
A* = 



K -K 



(K tol -K to0 )2-K, 



Letting i? sum denote the optimal sum rate, the two corner points in Figure El are 

1 



log , 
2 Pi 



KJ 1 |K g | 



Pi 



and 



5, 



1 , IKJ 1 , IKJ 

Rsum log r=r , , - log — — 

2 D 2 2 D 2 
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3 Lower Bound 



By fairly procedural steps, we have the following lower bound to the sum rate of the 
multiple descriptions: 



L 

n 

1=1 1=1 i=i 



L 



(8) 



= J2H(Q) - H(C ir ■ ■ ,C L ) + H(C U C L )-H(C h C L |x" 
i=i 

=I(C 1 ;C 2 ;...;C L )+I(C 1 , C L ;x"), 
where we have defined 

L L 

J(Ci; C 2 ; . . . ; C L ) = ^ (C,) - #(Ci, . . . , C L ) = ^ /(C,l d . . . CU), 

and called it the symmetric mutual information between C\, . . . , Cl- Note that 
I{Cx\ C 2 ; . . . ; Cl) > and is also well defined even when C 1 , . . . , are continuous 
random variables. Our main result is the following information theoretic inequality which 
gives a lower bound to the sum of symmetric mutual information between (Ci, C 2 , . . . , Cl) 
and mutual information between Ci, C 2 , . . . , Cl and x n for given covariance constraints. 

Lemma 1. Lei x n = (x[l], . . . , x[n]) ; where x[m] 's are iid. A/"(0,Ka.) Gaussian ran- 
dom vectors for m = 1, . . . , n. Lei Ci, . . . , Cl be random variables jointly distributed 
with x n . Let Xq = E[x n |Ci, . . . , Cl] and x™ = E[x n |C;] /or 1 = 1, . . . , Given positive 
definite matrices Di, . . . , D^, Do, i/ 



1 n 

- VE[(x[m]-x z [m]) t (x[m]-x / [m])] ^D z , / = 1, . . . , L, 
n / — ' 

x T 0) 

- ^ E[(x[m] - Xo[m])*(x[m] - x [m])] ^ D , 

/(C' 1 ;C 2 ;...;C i )+/(C 1 , C L ;x")> sup g log |K *' |K * + ^fOPg + K *' . (1 ) 

K "° |D |n|D ; + K,| 

Furthermore, there exists a jointly Gaussian distribution of (Ci, . . . , Cx, x n ) such that the 
inequality in (JTUJ) is tight. 
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This is a fundamental information-theoretic inequality which involves only the joint 
distribution 4 between Ci, C 2 , . . . , Cl and x n and bounds on mean square error estimation 
of x n from Ci, C 2 , ■ ■ ■ , Cl', we delegate the proof of this result to Appendix |B1 We can 
now use Lemma Q to derive a lower bound to the sum rate 

1, \K X \\K X + K Z \^\B + K Z \ 

2_^Ri> sup - log - . (11) 



1=1 



|D |n|D* + K, 

i=i 



By letting L = 1 in the lemma above, we can derive a simple lower bound to the rate 
of the individual descriptions as well: 

Ri > -H(d) = -{Hid) - H{Q\-x n )) 
n n 

= -/(x n ;C,) (12) 
n 

> - log rzr , , 1 = 1, ■ ■ ■ , L. 

This bound is actually the point-to-point rate-distortion function for individual receivers, 
since each individual receiver only faces a point-to-point compression problem. 

Note that for any positive definite K z , 

1 \K X \\K X + K Z \( L -V\-D Q + K Z \ 

2 ^g - L 

|D |n|D« + K z | 

i=i 

is a lower bound to the sum rate of the multiple descriptions. Two special choices of K 2 
are of particular interest: 

• Letting K z = el and e — ► + , we have the following lower bound: 

(13) 







L 


|Di 




|Dl| 



1=1 

This bound is actually the summation of the bounds on the individual rates. 
Letting some eigenvalues of K z goes to infinity, we have the following lower bound: 

L 



> Ri> - log — — T 
^ 2 6 D 

i=i 1 U| 



(14) 



4 This inequality holds even when C\, C2, . . . , Cl are not simply functions of x" and can also be 
continuous random variables. 
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This bound is the point-to-point rate-distortion function when we only have the 
central distortion constraint. 

We will see later that for some distortion constraints (D 1; . . . , D L , D ), (fTHj) and (fTlj) 
can be tight. 

4 Upper Bound 

In the previous section we gave a lower bound to the sum rate. Now we give a upper 
bound to the sum rate by using the jointly Gaussian description scheme described in 
Section O 

4.1 Jointly Gaussian Multiple Description Scheme 

First we give a sketch of the achievable rate region by using jointly Gaussian description 
scheme. Given the source sequence x n , as long as we can find a combination of codewords 
(u™, . . . , u£) that are jointly typical with x n , all the receivers can generate reproduction 
sequences that satisfy their given distortion constraints. An intuitive way to understand 
(0) is the following: since (u™, . . . , uj) are jointly typical with x™, then for any S C 
{1, . . . , L}, we have that u™, I G S are jointly typical with x n . Now the probability that 
a randomly generated combination of codewords u™, I G S are jointly typical with x" is 
roughly 

gfi/i(uj ,Zs5|x) 
J~J e nh(ui) ' 

and the number of possible combination of codewords u",Z G S are Yl e nRl . Thus, as 
long as 

les les 

we can find a combination of codewords u", I G S that are jointly typical with x n . 
Rigorously speaking, we need to show that as long as (j!5|) is satisfied, then for any 
given source sequence x" we can find a combination of codewords (u", . . . , u£) such 
that uf,l G S are jointly typical with x" for all S C {1, . . . , L}. The second moment 
method [21] is commonly used to address this aspect, and a proof can be found in [7]. 

Evaluating (|15J) based on the jointly Gaussian distribution of x and Ui, . . . , u^, we 
get that all the rate tuples (i?i, . . . , Rl) satisfying 

niK.+K^i 

J2 R i>J2 h ( n ti- h ( ni ' leS W = o l °Z — IK I ' WC{1,...,L} (16) 
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are achievable by the jointly Gaussian description scheme. In particular, we have that 
the achievable sum rate is 



L 

m i 

H u i) - h( n u ■ • • > u ^l x ) = 9 lQ g ' =1 , K | • ( 17 ) 



1=1 



The resulting distortions (D*, D^, Dq) by using jointly Gaussian description scheme 
can be calculated as 

Dr =Cov[x|u ; ] = [k; 1 + k- 1 ]- 1 , I = 1, . . . , L, 
D*=Cov[x|u 1; u L ] = [K; 1 + (I, I)K W \I, I)*]- 1 . 

4.2 Combinatorial Property of the Achievable Region 

The achievable region given in (|T3j) has useful combinatorial properties; in particular 
it belongs to the class of contra-polymatroids [22]. Certain rate regions of the multiple 
access channel [23] and distributed source coding problems [24] are also known to have 
this specific combinatorial property. To see this, let 



<t>{S) = Y,h(ui)-Kui,leS\x), SC{1, L}. 



def 

les 

We can readily verify that 

0(£U{t})>0(S% VtG{l, L}, 
0(SUT) + 0(SnT) > (f>(3) + <f>(T). [ ' 

By definition, we conclude that the achievable rate region of a jointly Gaussian multiple 
description scheme is a contra-polymatroid. The key advantage of this combinatorial 
propety is that we can exactly characterize the vertices of the achievable rate region 
(JT3J). Letting it to be a permutation on {1, . . . , L}, define 

b i = Hi^l, 7T2, • • • , 7Ti}) - 0({7Ti, 7T 2 , . . . , 7Tj_i }) , % = 1, L, 

and = (b^\ . . . , b^j. Then the L\ points {b^, tt a permutation} are the vertices 
of the contra-polymatroid ()15|). 

4.3 Comparison of Upper Bound and the Lower Bound 

Our goal is to show that the jointly Gaussian description scheme achieves the lower 
bound to the sum rate. In general it does not seem facile to do a direct calculation and 
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comparison. We forgo this strategy and, instead, provide an alternative characterization 
of the achievable sum rate which is much easier to compare with the lower bound. 

Similar to the derivation of the lower bound (in Appendix [HJ), we consider an A/"(0, K. z ) 
Gaussian random vector z, independent of x and all w/'s. Defining y = x + z, we have 
the following achievable sum rate: 



5> 

1=1 



1=1 

L 

^2h(ui) - h(ui, 



, u L |x) 

, u L ) + h(ut, u L ) - h(ut, u L |x) 
h(ui) - h{u!, ■■■ ,u L ) + /(ui, . . . , u L ; x) 

1=1 

L / L 

> ^ H u i) ~ M u i> ■ ■ • , u l) + ^(ui, . . . , ul;x) - I h(ui\y) - h(u u u L \y) 
i=i \i=i 

L 

^2 W y ) ~ h (y\ u i)) - My) + Hy\ui, + h(x) - h(x\u l: . . . , u L ) 



(a) 



1=1 
L 



1=1 



h(x) + (L — l)h(y) - ^2 Ky\ n i) + h (y\ u i> • • • > u l) - h(x\ui, u L ) 



K, 



K. + K ? 



l=i 

\{L-1) 



Cov[x|ui, . . . , u L ] + K- 



Cov[x|ui, . .., ul] n Cov[x|u z ] + K, 
i=i 



(20) 



where the last step is from a procedural Gaussian MMSE calculation. 
Note that if we have 

L 



J^/i(u/|y) -/i(ui, u L |y) =0, 



(21) 



i=i 



then (a) in (j2T?j) is actually an equality. Thus, if our choice of K w and K z satisfy the 
following two conditions: 

• (|2T]) is true. 

• distortion constraints are met with equality, i.e., 

Cov[x|ui] = D h I = 1, . . . , L, 



Cov[x|ui, . . . , Ul] = D 



(22) 



ri- 



ll 



then the upper bound matches the lower bound and we have characterized the sum rate. 
In the following we examine under what circumstances the above two conditions are true. 

First, we give a necessary and sufficient condition for (J2*TJ| to be true, delegating the 
proof to Appendix O 

Proposition 1. There exists some choice of positive definite K. z such that (J21j) is true 
if and only ifK w , the covariance matrix of (w 1; • • • , wj, takes the following form 



fK Wl -A -A 
—A A 



K, 



A 



-A 



A K- U , L _ 1 A 
-A -A K W J 



where -< A -< K. 



(23) 



Next, we look at the conditions for (j22j) to be true. From (j!8|) . we have 
D- 1 = Covfxlur 1 = K; 1 + K- 1 , 1=1, L 



(24) 



Do 1 = Cov[x| Ul , . . . , ul]- 1 = K- 1 + (I, . . . , 1)K-\1, I)'. 

. , I)*, is calculated in the following lemma; the proof is 



(I, I, I)K^(I, I, 

available in Appendix ID1 

Lemma 2. Let 



K, 



fK Wl -A -A 
—A ~K. W2 A 



-A\ 
A 



-A ... 



A K 



U>L-1 



\-A ... -A -A K WL .J 



IfK w y and A y 0, then 

I)KZ 1 (I, I 



I) f 



^(K^+A)- 1 -A 



, i=i 



Using this lemma, from ()24|) we arrive at 



[(Do 1 - K; 1 )- 1 + A] - 1 = m 1 ~ K.T 1 )- 1 + A] 



(25) 



Defining 



K W0 = (Do 1 - K- 1 )- 1 , 



(26) 
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is equivalent to 

L 

[K W0 + Ap 1 = ^ [K^ + Ap 1 . (27) 
j=i 

Thus, if there exists a positive definite solution A to (|27jl . and the corresponding K w is 
positive definite, then the distortion constraints are met with equality, i.e., (J2*2*j) holds. 
It turns out that as long as A is a solution to (|27jl. the resulting K w is always positive 
definite; we state this formally below, delegating the proof to Appendix lEl 

Lemma 3. If for some K wo y and A >- (|'27jl inte, t/ien t/ie covariance matrix K w 
defined in (|23|) positive definite. 

We summarize the state of affairs in the following theorem. 

Theorem 3. Given distortion constraints (D 1; D^,D ) ; /e£ 

^^(D^-K- 1 )- 1 , / = 0, 1, . . . , L. (28) 

// taere exists an solution A* to (J27)) and -< A* -< K^., iaen iae jointly Gaussian 
description scheme with K w defined in (J23|) A = A* achieves the optimal sum rate, 
and the optimal K z /or lower bound flllj) K 2 = K X (K X . — A*) -1 !^ — K s . 

Thus we show that if the given distortion constraints (D 1; D^,D ) satisfy the 
condition for Theorem then the jointly Gaussian description scheme achieves the op- 
timal sum rate and we can calculate the optimal K w by solving a matrix equation. 
However, for arbitrarily given distortion constraints, (|27jl may not have a solution A* 
such that -< A* -< K^. In this case, we can show that there exists a jointly Gaussian 
description scheme that achieves the sum rate lower bound, and resulting in distortions 
(D*, . . . , D£, Dp) such that D^* Dj for I = 0, 1, . . . , L. In the following we first study 
the relatively simpler case of scalar Gaussian source, and then move to discuss the vector 
Gaussian source. 



5 Scalar Gaussian Source 



Here we suppose that the information source is an i.i.d. sequence of jV(0, cr^) scalar 
Gaussian random variables. Let individual distortion constraints be (di, . . . , di) and 
the central distortion constraints be do, where < do < di < a\ for I = 1, . . . , L. We 
consider the jointly Gaussian description scheme with the following covariance matrix 
for wi, . . . , wi. 



( 



07 



-a at, 



K, 



—a 
\—a 



-a 
-a 



—a 

—a 



a 



L-l 

a 



—a\ 
—a 

—a 

a?, 



(29) 
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Consider the condition for Theorem El to hold: to meet the individual distortion con- 
straint with equality, we need 

o? = ft 1 - v?)- 1 = 1 = 1, ...,L. (30) 

Let 

J2 def / ,-1 -2\-l do(J x . . 

C7 = (d Q - cr x ) = — — , (31) 

a x - a 

we need 

K + ^ 1 = EK + ^ 1 ( 32 ) 

to have a solution a* G (0,cr^), to meet the central distortion constraint with equality. 
Towards this, define 

L 



/W = ^-E^. (33) 
and we have 

1 A 1 1 L — 1 Al 

/(O = ^-S^ = 3(S*"*" (L_1)0SI " 



(34) 



Using induction, we can show that 



L - - 1 



Z=l 

Thus we have 

/(0) < =>■ /(<# < 0, 
/(<# > =► /(0) > 0. 

Then given distortions (di, d^, d ), /(0) and /(of) falls into the following three 

cases. 

Case 1: /(0) > and /(of) < 0. 

In this case, since f(a) is a continuous function, there exists an a* G (0, o^) such that 
/(a*) = 0. In this case the condition for Theorem El holds and from Theorem El we know 
that jointly Gaussian description scheme with covariance matrix for wi, . . . , wi being 
(|29|) with a = a* achieves the optimal sum rate. 
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L 

Case 2: f(0) < 0. Alternatively, JL + ^_£-L<0. 

In this case, the condition for Theorem |3] does not hold. But the jointly Gaussian 
description scheme can still achieve the sum rate. To see this, choosing a = in K. w we 
can meet individual distortions with equality and get a central distortion d' . From (J23I) 
we have 



°x l=l <I\ l=1 a l °x 
1 

> — . 

d 

Hence we have achieved distortion (di, . . . ,di,d' ) where d' < do, and from (|17jl the 
achievable sum rate is 

L ^ 2L 

S^^s^sfe- (37) 

which equals the sum of our bounds on individual rates. 

L 

Case 3: f(a 2 x ) > 0, Alternatively, XI dl ~ do - (L - l)o% > 0. 

l=i 

In this case, the conditions for Theorem|3]do not hold as well. But the jointly Gaussian 
description strategy still achieves the sum rate. To see this, note that we can find a d' L 
such that < d' L < di and 

L-l 



J2di + d' L -d -(L-l)a 2 x = 0, (3* 



L ~ u x 



and we choose a = a 2 , af = (d l 1 — a x 2 ) 1 for I = 1, • • • , L — 1, and a\ = (d' 
in K w . Defining <Xq = (d$ 1 — cr^T 2 ) -1 , (jHSj) is equivalent to the following equation: 

K+^r^EK+^r 1 - (39) 
i=i 

From Lemma H3 our choice of K w is positive definite. Thus the resulting distortions are 
(di, . . . , d L -i, d' L , d ), where < d' L < d L . 
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Using the determinant equation 



1=1 1 x 1=1 



and (|39|). we have an achievable sum rate 

We conclude that in this case the point-to-point rate-distortion bound for the central 
receiver is achievable. 

In summary, we have shown that the jointly Gaussian description scheme achieves the 
lower bound on the sum rate. Further, the sum rate can be calculated either trivially 
(by choosing a* = in case II or a* = 1 in case III) or by solving a polynomial equation 
in a single variable (case I). 

6 Vector Gaussian Source 

The essence of our proof of the optimality of jointly Gaussian description scheme for 
scalar Gaussian sources is the use of the intermediate value theorem for scalar continuous 
functions. However, there is no natural extension of this theorem for vector valued 
functions. To avoid this problem, we first explicitly solve the two description problem and 
characterize the optimality of jointly Gaussian description scheme. Next, we show that 
the jointly Gaussian description scheme is optimal for L > 2 by showing an equivalence 
of certain optimization problems. In the last part of this section, we show that the jointly 
Gaussian description strategy can achieve the optimal rate region for the two description 
problem. 

6.1 Explicit Solutions for Some Cases of Two Description Prob- 
lem 

With only two descriptions, we can explicitly solve (J27J) , thus generalizing the corre- 
sponding solution for the scalar Gaussian source, derived in [1]. 
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-o* 


-ol ... 


-°l 








-a* ... 




-°l 




o\ 


-ol ... 




-°l 




~°l 


~°l °L-1 


-ol 








1 1 


o\ 



(41) 



Suppose the distortion constraints are denoted by (Di, D 2 , D ) and let 

iz _ / —A* 
»*» ~ ^_ A * K^ 2 

We now solve (J2H), which is equivalent to (|27jl. for K TOl , K„ 2 and A*. From (J23J) we get 

^^(D^-K- 1 )- 1 , I = 1, 2, (42) 

and 

Do^K^ + ai^^II)*. (43) 
Expanding out K^ 1 using Lemma El in Appendix [XJ we get 

Dp 1 - K; 1 = K^j + (I + K^A*)(K W2 - A*K£A*)- l (I + A*K~J). (44) 

Taking inverse on both sides, we have 

(Do 1 - K; 1 )- 1 = K W1 - (K W1 + A*)(K W1 + K W2 + 2A*)- 1 (K Wl + A*). (45) 

Defining as 

K^D^-K- 1 ]- 1 , (46) 

(|43|) is equivalent to 

K m - K wo = (K W1 + A*)(K W1 + K W2 + 2A*)-\K W1 + A*). (47) 

Defining 

X^K^+A*, 

is equivalent to 

K W1 - K W0 = X(2X + K W2 - K Wl )~ l X, (48) 

which is further equivalent to 

X(K W1 - K^ )- X X = 2X + K W2 - K W1 . (49) 

This is a version of the so-called algebraic Riccati equation; the corresponding Hamiltonian 
is readily seen to be positive semidefinite and we can even write down the following 
explicit solution: 



X K W() 

+ (K W1 - K m )i (K Wl - K W0 )-i(K W2 - K Wo )(K m - K r" 



2 (K W1 - K m )2. 



(50) 
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Thus 

(51) 

Now, if -< A* -< K. x then we can appeal to Theorem |3] and arrive at the explicit 
jointly Gaussian description scheme parameterized by K. w that achieves the sum rate. 
Analogous to the scalar case (cf. [1]), we have the following sufficient condition for when 
this is true; the proof is available in Appendix IF1 

Proposition 2. If the distortion constraints (Di, D2, Do) satisfy 

D + K x - D x - D 2 y 

and D 1 + K; 1 - D^ 1 - D 2 1 y 0, { ' 

then -< A* -< K x . 

We now complete the proof by considering the cases that are not covered by the con- 
ditions in Proposition EJ 

• When 

D 1 + k; 1 - Dr 1 - D 2 1 ^ 0, 

we can choose A* = to achieve the sum of point-to-point individual rate-distortion 
functions. Thus in this case, the sum rate is equal to this natural lower bound. 

• When 

D + - Dx - D 2 ^ 0, 

we can choose A* = K. x to achieve the point-to-point rate distortion-function for 
central receiver, also a natural lower bound. 

• When neither Do + K x — Di — D 2 nor Dq 1 + K^; 1 — D^ 1 — D 2 1 is positive or 
negative semidefinite (this case cannot happen in the scalar case), we cannot use 
Theorem El and the trivial choice of A* = or A* = K x does not meet the lower 
bound. In the next subsection we will address this case and prove that the jointly 
Gaussian description scheme indeed achieves the lower bound on the sum rate for 
L>2. 

If we let the source to be scalar Gaussian, our result reduces to Ozarow's solution of 
the two description problem for a scalar Gaussian source [1]: this is because the last case 
described above does not happen in the scalar case. 



K^-K 



WO/ 



Wl 



K 



WO J 



(K 



«>2 



K W0 )(K W1 -K 



Wo I 
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6.2 Solutions for L > 2 

While we exactly characterized the optimal jointly Gaussian description scheme and 
used this characterization in arguing that it achieves the fundamental lower bound to 
the sum rate, such exact calculations do not appear to be as immediate when L > 2. So, 
we eschew this somewhat brute-force approach and resort to a more subtle proof that 
involves exploring the structure of the solution to an optimization problem. First, note 
that by a linear transformation at the encoders and the decoders, we have the following 
result on rate region for multiple description with individual and central receivers. 

Proposition 3. 

R*{K X , D 1? ,D L , D ) = K^DxK^, ,K~*T> L K~\ K^DoK ' 



2 ' 

X 



(53) 

Thus, throughout this subsection we will suppose, for notation simplicity, that K x = I. 
Given distortion constraints (Di, ... D^, D ), let 

K,, = (Dr 1 - I)" 1 , / = 0, 1, ...,L, (54) 



and define 



Note that 



/(A) d ^ f [K W0 + A]" 1 - £ [K Wi + A]" 1 , (55) 

i=i 

F(A)= log|K W0 +A|-Elog|K to! + A|. (56) 

i=i 

Consider the following optimization problem: 

max F(A). (58) 

Now, since F(A) is a continuous map and =<! A =<! I is a compact set, there exists an op- 
timal solution A* to (J58|) where A* satisfies the Karush-Kuhn- Tucker (KKT) conditions: 
there exist Ai )p and A 2 >p=- such that 

/(A*)+Ai-A 2 = (59) 
A X A* = (60) 
A 2 (A*-I) = 0. (61) 

Now A* falls into the following four cases. 
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Case 1: -< A* -< I. Alternatively, and 1 are not eigenvalues of A*. In this case, 
Ai = and A 2 = 0; thus the KKT conditions in (JoUj) reduce to 



/(A*) 



0. 



Equivalent ly, 



[K, 



AI 



A' 



i-i 



(62) 



i=i 



From Theorem |3J the jointly Gaussian description scheme with covariance matrix for 
wi, . . . , W£ being 



fK m -A 
-A* K 



«'2 



-A* 
-A* 



-A*\ 
-A* 



-A* 
\-A* 



-A* 
-A* 



K 



WL-l 

A* 



-A* 



K, 



(63) 
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achieves the lower bound to the sum rate. Thus in this case, we have characterized the 
optimality of the jointly Gaussian description scheme parameterized by (JBlIj) in terms of 
achieving the sum rate. 

Case 2: ^( A* -< I. Alternatively, some eigenvalues of A* are 0, but no eigenvalues 
of A* are 1. Thus A 2 = and the KKT conditions in (jBT?|) reduce to 



(K 



U<() 



+ A*)- 1 -^(K^+A*)- 1 + A 1 



0, 



(64) 



i=i 



for some Ai ^ satisfying AiA* = 0. The key idea now is to see that the distortion 
constraint on the central receiver is too loose and we can in fact achieve a lesser distortion 
(in the sense of positive semidefmite ordering) for the same sum rate. We first identify 
this lower distortion: defining 

k; o = (kzI + a 1 )-\ 

consider the smaller distortion matrix on the central receiver 

-1 . l T . T .-1 . » \-l 



D 



o 



(K^+ir = (i +k;;+ AO- 



Do 1 



Ax)" 1 < D . 



This new distortion matrix on the central receiver satisfies two key properties, that we 
state as a lemma (whose proof is available in Appendix lG*|) . 



Lemma 4. 



[K W0 + A* 



Ai 



IDq + K, 

IDnl 



(k; + a* 

|DS + K Z | 
IDS I 



(65) 
(66) 
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Comparing ()64j) with (|65|). we have 

L 

[K^ + A^^^fK^+A*]- 1 . (67) 
1=1 

Now, the corresponding K 2 = (I — A*) -1 — I is singular. If it hadnt been, then by 
Theorem |H] we could have concluded that jointly Gaussian description scheme achieves 
the lower bound to the sum rate. We now address this technical difficulty. 

Our first observation is that there exists 5 > such that for all e G (0, 6) we have 
-< A + el -< I, and -<! K* o - el, ^ K Wl - el, and we can rewrite (jHZ|) as 

L 

[(K; o - el) + ( A* + el)] 1 = [( K -< - el) + ( A* + el)] - 1 . (68) 

i=i 

Thus if the distortion constraints were (D^e), . . . , D L (e), D (e)) with 

D,(e) = [{K Wl -eiy l + l]~\ 1 = 1, L, 
r> (e)=[(K* Wo -eI)- 1 + l}-\ 

then A* + el is a solution to (fo^j). This situation corresponds to that discussed in Case I; 
we can conclude that sum rate for this modified distortion multiple description problem 
is 

l log |I + K, M p->|D ( £)+ K, M | {m) 
|D„( e )|n|D,(e)+K,(e)| 

1=1 

where K z (e) = [I — (A* + el)] -1 — I. We would like to let e approach zero and consider 
the limiting multiple description problem. In particular, we show that 

D,(e) -> B h 1 = 1, L, (70) 
Do(e) - Dq, (71) 

as e — > in Appendix El Further, we show that 

K^eJ-CI-AT 1 -!, (72) 
as e — > in Appendix |TJ Thus we can conclude that the sum rate approaches, using 



1 II + KsI^IDq + KsI 

2 lo § 7^ » ( 73 ) 

iDoiniD^ + K,! 

i=i 

as e — > 0; here K z = (I — A*) -1 — I. We observe that this sum rate is achievable using 
the jointly Gaussian multiple scheme. Further, this sum rate is identical to the lower 
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bound to sum rate for the original distortions (Di, . . . , D^, D ). Thus we conclude the 
optimality of the jointly Gaussian description scheme in this well. 

Case 3: -< A* ^ I. Alternatively, some eigenvalues of A* are 1, but no eigenvalues 
of A* are 0. In this case, the Ai = and the KKT conditions in (JoTJj) reduce to 

L 

(K W0 + A*)' 1 - J](K W; + A*)- 1 - A 2 = 0, (74) 
i=i 

for some A 2 >p satisfying A 2 (A* — I) = 0. Defining 

K^=[(K Wl +T)- 1 +A 2 ]- 1 -I i 

we have, as in that 

(K Wi + A*)" 1 + A 2 = (K; ; + A*)" 1 . (75) 

The observation 

(K Wi + A*)- 1 + A 2 = [(K W! + 1) + (A* - I)]" 1 + A 2 , 
combined with the proof of suffices to justify (J7SJ). Now, from (|75)i. 

L-l 

(K W0 + AT 1 ~ £( K -< + A*)" 1 - (K; l + A*)- 1 = 0. (76) 
i=i 

As in the previous case, the key step is to identify smaller distortion matrices at each of 
the individual receivers (ordered in the positive semidefinite sense) that is achievable at 
the same sum rate: 

d;= [kj^ + i]- 1 , 1 = 1, ...,L. 

To see that this is indeed a smaller distortion matrix, observe that since K w is positive 
definite, it follows that y and 

d;= [K^ + If 1 

(K W! + I)- 1 + A 2 )~ 1 -l) -1 + I 

= [I - (K m + 1)- 1 - A 2 ] 
= [I + K^-A, 
= D Z - A 2 , 1 = 1, L. 

Since A 2 y- 0, it follows that -< ^ D t , 1 = 1, L. Define 

D,(e)= [(K wi + el)- 1 + l] _1 , 1 = 0, 1, . . . , L — l, 
r> L (e)=[(Kt L + eI)- 1 +l}-\ 

22 



(77) 



(7f 



then there exists 5 > such that for all e G (0, 5) we have -< A* — el -< I, and 
-< T>i(e) -< I. We can rewrite (fTSj) as 



L-l 

i -l 



[(K wo + el) + (A* - el)]" 1 = £ [(K Wl + el) + (A* - el)}'^ [(K* Wl + el) + (A* - el 

i=i 

(79) 

Thus if the distortion constraints were (Di(e), . . . , Dx(e), D (e)), then A* — el is a 
solution to (|T9*j) . This situation corresponds to that discussed in Case I; we conclude that 
the sum rate for this modified distortion multiple description problem is 

l log |I + K,( e) |(^|D (e) + K,(e)| (g0) 
|D„(e)| n|DiW+ K»(e)| 

1=1 

where K 2 (e) = [I — (A* — el)] -1 — I. We would like to let e approach zero and consider 
the limiting multiple description problem. Similar to equations (fTHJ) and (JHJ), we have 

D,(e)-.D,, 1 = 1,. ..L, 

d„( £ ) - d;. (81) 

Further, we show that 

|I + K,W|^-)|Do(Q + K, W | =1 (g2) 
n|D,(e)+K s ( e )| 

1=1 

in Appendix \j\ We can now conclude that the sum rate approaches 

l los ik (83) 

as e approaches 0. In other words, the point-to-point rate-distortion function for central 
receiver with distortion Do can be achieved by using the jointly Gaussian description 
scheme, and the resulting distortion is (Di, D L , D ) where -< D* L D/,. In 
conclusion, the jointly Gaussian description scheme is also optimal in this case. 

Case 4: ^ A* =<! I. i.e., both and 1 are eigenvalues of A*. In this case, the KKT 
conditions are: there exist A x )p and A 2 )p such that equations (|59*|). (|5U|l and (j^Tjl 
hold. We can combine equations (jSSj) and (|75|) to get 

L-l 

(K; o + A*)- 1 = £(K W; + A*)- 1 + (K* WL + AT 1 , (84) 
i=i 

where 

K^ = (K^ + A 1 )- 1 > 

K* WL = [(K t0L +I)- 1 + A 2 ]- 1 -I. 
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As in cases 2 and 3, we want to show the optimality of the jointly Gaussian multiple 
description scheme through a limiting procedure. We do this by first perturbing A* so 
that it has no eigenvalue equal to or 1 as follows. 

Without loss of generality, suppose that A* has p eigenvalues equal to and q eigen- 
values equal 1, where p > and q > 0, and there exists N x N orthogonal matrix Q such 
that 

QA*Q* = diag{ 0, , 1, 1 , a p+q+1 , . . . , a N }, 
v g 

with < a p+q+ i < 1, . . . , < ajv < 1. We need to perturb the eigenvalues of A* away 
from both and 1. Towards this, we define two N x N diagonal matrices: 



Ei = diag(l, 



1, 0, 0, 0, 



diag(0, 




Also define 



A*(e 1 ,e 2 )=A* + Q t (e 1 E 1 -e 2 E 2 )Q, 
K 4 (e 1 ,e 2 ) = (I-A*(ei,e 2 ))- 1 -I, 
K^( ei ,e 2 ) = K Wl -Q t (e 1 E 1 -e 2 E 2 )Q, I 
K WL (e u e 2 ) = K* WL - Q t (e 1 B 1 - e 2 E 2 )Q, 
K W0 {e u e 2 ) = K* - Q'^Ex - e 2 E 2 )Q. 



1, 



Further, defining 



D,(ei,e 2 ) = (I + ^(ei, ez))" 1 , 1 = 1 



15) 



there exists S > such that for all e\ G (0, 5) and e 2 G (0, 5) we have -< A*(ei, e 2 ) -< I, 
and -< D|(ei, e 2 ) -< I. Now, we can rewrite (IMI) as 



K, 



ei,e 2 ) + A*(ei,e 2 )j = [K ttl (ei,e 2 ) + A*(ei, e 2 



i=i 



36) 



Thus if the distortion constraints were (Di(ei,e 2 ), D^(ei,e 2 ), D (ei,e 2 )), then 

A*(ei,e 2 ) is a solution to (|S6^1. This situation corresponds to that discussed in Case I; 
we conclude that the sum rate for this modified distortion multiple description problem 
is 

1 |I + K z (e 1 , e2 )|( i - 1 )|D (e 1 ,e 2 ) + K 2 (e 1 ,e 2 )| 



■log- 



(87) 



|Do(e l5 e 2 )| n |Di(ei,e 2 ) + K a (ei,e 2 ) 
i=i 
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where K z (ei, e 2 ) = [I — A*(ej., 62)]" —I. We would like to let e\ and e 2 approach zero and 
consider the limiting multiple description problem. Similar to equations (|70|) and (JHJ), 
when ei and e 2 approach 0, we get 

D,(ei,e 2 )->D,, 1 = 1, L-l, 

D i (e 1 ,e 2 )^D2, (88) 
D (ei,e 2 ) -> Dq, 

where = — A 2 as in case 3 and Dq = [Dq 1 + A^ 1 ] -1 as in case 2. Further, we 
show that 

r 1, |I + K 2 ( £l , e 2 )|( L - 1 )|D (e 1 , e 2 ) + K 2 (e u e 2 )\ 1 1 

llm n llm n l0g Z = l0g TFTl ( 89 ) 

|D (e 1 ,e 2 )|n|D / (ei,e 2 )+K 2 (e 1 ,e 2 )| 1 01 

i=i 

in Appendix |K] We conclude that the sum rate approaches 

l hg jk\ (90) 

as €1 and e 2 approach 0. Thus the point-to-point rate-distortion function for central 
receiver with distortion Do can be achieved by using the jointly Gaussian description 
scheme, and the resulting distortions are (Di, . . . , D^, DJ$) where -< D^ ^ D^ and 
-< Dq =^ D . In other words, the jointly Gaussian multiple description scheme is also 
optimal in this case. 

To summarize, we see that the jointly Gaussian description scheme achieves the limiting 
sum rate. The limiting sum rate is the solution to an optimization problem. For some 
specific distortion constraints, the sum rate can be characterized as the solution to a 
matrix polynomial equation (Case I). 



6.3 Rate Region for Two Descriptions 



Applying the result in Section 16721 to the case of L = 2, i.e., the two description problem, 
we can see that jointly Gaussian description scheme achieves the optimal sum rate. This 
resolves the case left out in Section 16.11 It also turns out that in the two description 
problem, we can show that jointly Gaussian description strategy achieves the entire rate 
region. This is the main result of this subsection. 

From SectionElwe have a outer bound to the rate region for the two description problem 

R 2 ) : 



Tl 0Ut {K x , D ls D 2 , D c 



1 KJ 

Ri> - log 7=77 

2 6 D, 



1 = 1,2 



i?i + R 2 > sup - log 
K z yO 2 



|K*| 


|Kz + K z | 


|D + K Z | 


|Do| 


Di + K z \ 


|D 2 + K 2 | 



>■ (91) 
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Following the discussion in Section lfi~2*l we show in the following that the jointly Gaus- 
sian description strategy (jointly Gaussian multiple description schemes and the time 
sharing between them) achieves the outer bound to the rate region. 



Let 
and 



K W! = (D- 1 -K; 1 )- 1 > 1 = 0,1,2 



F(A) = log \K W0 + A| - log \K W1 + A| - log \K W2 + A| 
Now consider the optimization problem: 

max F(A). 
As in Sectioning the optimal solution A* falls into four cases. 



(92) 



Case 1: -< A* -< K. x . In this case, we know from Section 0] that the rate pair 
(RijRz) satisfying 



(Rl, R2) : 
Ri > - log 



|K X + K 



Wl I 



IK 



I = 1, 2 



Ri + R2 > 2 lof 



IK^ + K^J 


IK^ + K^I 









(93) 



is achievable using the jointly Gaussian multiple description scheme with the covariance 
matrix of wi , W2 being 

K„i, A 



K, 



A* K 



IV 2 



Denoting the resulting distortions as (D 1; D 2 , D ), we readily calculate 



IK. + K 



w, I 



IK 



JloglK.HK^ + K^H Jlog 



WJ I 



|K a .| 
ID/I 



for I — 1, 2. From the discussion in Section 1672} we know that the lower bound to sum 
rate is achieved using this jointly Gaussian description scheme. Thus, in this case, the 
jointly Gaussian description scheme achieves the rate region. As an aside, we note in this 
case that, A* satisfies 

[K; o + A*] - 1 = [K W1 + A*]" 1 + [K W2 + A*} , 
and, from the discussion in Section KTT\ that a sufficient condition for this case to happen 

is dm. 

Case 2: =<! A* -< K^. This case is similar to case 1: the jointly Gaussian description 
scheme with covariance matrix for wi, W2 being 



K 7 



K —A* 
A 
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achieves the lower bound on the rate region. We note that in this case the resulting 
distortions are (Di, D 2 , Dq), with Dq =<! D . Further, we know from the discussion in 
16.11 that a sufficient condition for this case to happen is 

D 1 + K- 1 - Dr 1 - D 2 1 ^ 0. 



Case 3: -< A* ^ K x .. In this case, we know from the discussion in Section l6~2*l that 
for another two description problem with distortions (Di, D?i, D ) such that ^ D 2 , 
the jointly Gaussian description scheme with covariance matrix for w 1; w 2 being 

tv- _ ( K Wl — A*\ 

achieves the lower bound to sum rate log j^M to the original distortions (D 1; D 2 , D ). 

We can see, from the contra-polymatroid structure of the achievable region of jointly 
Gaussian description scheme, that the corner point 





|K*| 




|K*| 


1, |K a |\ 


G iog 




\ log 








|Di| 




|Do| 


-2 l0g | Dl |J 



in Figure El is achievable by this jointly Gaussian description scheme. 

Now observe that the discussion in case 3 of Section 16.21 is symmetric with respect to 
the individual receivers. Thus, by exchanging the role of receiver 1 and receiver 2, we 
can achieve the other corner point 

fl \K X \ 1 |K,| 1 \K X \\ 

B 2 = - log — — log — — - , - log — — - 

V2 |D | 2 S |D 2 |'2 & \B 2 \J 

in Figure El by another appropriate jointly Gaussian description scheme. Finally, time 
sharing between these two jointly Gaussian multiple description schemes allows us to 
achieve the lower bound on the rate region. As an aside, we note, as a consequence of 
the discussion in Section 16.11 that a sufficient condition for this case to happen is 

D + K x - Dx - D 2 ^ 0. 



Case 4: ^( A* ^ K^. In this case, we know, from the discussion in Section IB~2*l that 
for another two description problem with distortions (Di, D 2 , Dq) such that D 2 ^ D 2 
and Dq =^ Do, the jointly Gaussian description scheme with covariance matrix for wi, w 2 
being 



K 



K — A* 



-A* K* 

achieves the lower bound to sum rate ^| log j^jj to the original distortions (Di , D 2 , D ) 
Using an argument entirely analogous to that applied that the jointly Gaussian descrip- 
tion strategy achieves the rate region. 
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To summarize: the jointly Gaussian description strategy achieves the rate region for 
the two description problem. For a class of distortion constraints, the corner points of 
the rate region can be characterized by solving a matrix polynomial equation, as already 
seen in Section ItTTl 

7 Discussions 

Although multiple description for individual and central receivers is a special case of the 
most general multiple description problem, the solution to this problem sheds substantial 
insight to the issue- at- large. In this section, we discuss two instances of other multiple 
description problems that can be resolved using the insights developed so far. In partic- 
ular, we discuss the problem of two descriptions with separate distortion constraints and 
the general multiple description problem for some special sets of distortion constraints. 

7.1 Two Description with Separate Distortion Constraints 

The problem of two descriptions with separate distortion constraints is ilustrated in 
Figure El Suppose the vector Gaussian source x[m] = (xi[m],x 2 [m]), the dimension of 
xi[m] is Ni and the dimension of x 2 [m] is N 2 . This implies that the dimension of x[m] is 
N = N x + N 2 . Let K x = E[x[m]*x[m]], K X1 = Efx^mpx^m]], and K X2 = E[x 2 [m]*x 2 [m]]. 
There are two encoders at the source providing two descriptions of x[m]. There are three 
receivers: the individual receivers 1 and 2 are only interested in generating reproduction 
of Xi[m] with mean square distortion constraint Di (an N\ x N± positive definite matrix) 
from description 1 and x 2 [m] with mean square distortion constraint D 2 (an iV 2 x iV 2 
positive definite matrix) from description 2, respectively. The central receiver uses both 
descriptions to generate a reproduction of x[m] with the error covariance meeting a 
distortion constraint D (an N x N positive definite matrix) from both descriptions. 

This situation is closely related to the two description problem and we can harness our 
results thus far to completely characterize the rate region of the problem at hand. 

Theorem 4. The rate region of two description with separate distortion constraints is 

ft(D ls D 2 , D ) = |J K*(D[, D' 2 , D ), (94) 

T(D[,-D> 2 ) 

where T(D' 1 , D' 2 ) is defined as 

T(Di, D' 2 ) d = f { (Di, D' 2 ) : (r>[) {1 ,..., Nl} 4 D 1; (B' 2 ) {Nl+1 _ N} ^ D 2 } . (95) 



2$ 



Proof. It is clear that any rate pair (Ri,R 2 ) G ^(D^, D' 2 , D ) for some (D^, D' 2 ) G 
T(D' 1 , D 2 ) is in the rate region for the two description with separate distortion con- 
straints, and so 

7^(Di, D' 2 , D ) C K(Di, D 2 , D ). 

On the other hand, although receiver 1 (2) is only interested in reconstructing xi (X2), 
they can actually reconstruct the entire source x based on their received descriptions. 
Hence, any coding scheme for the two description with separate distortion constraints 
will result in some achievable distortions (D' 1; D' 2 , D' ) with (D' 1; D' 2 ) G T(D' 1 , D' 2 ) 
and D =4 Do. Thus any rate pair (Ri,R 2 ) G 1Z(Di, D 2 , D ) achieved by this coding 
scheme is in the rate region 72.* (D^, D' 2 , D ) for the two description problem. Thus 

ft(Di, D 2 , D ) C (J ft*(D' l5 D' 2 , D ). 

T(Di,Di) 

From equivalence of the two regions in ()94jl . the proof is now complete. □ 



7.2 General Gaussian Multiple Description Problem for Special 
Choices of Distortion Constraints 



Consider the general Gaussian multiple description problem with source covariance K x 
and 2 L — 1 distortion constraints D$ for each S C {1, . . . , L}. 

Following arguments similar to that used in arriving at the lower bound (jll)) for sum 
rate, we have an outer bound on the rate region: 



T^outfKa;, Di, D L , D ) 



(Ri, . . . , Rl) ■ 

2^Rl> 2 i0 S IDcl TT ID.+KJ ' V^Gjl, 



|D S | n |D ; +K Z 

les 



(96) 

Following arguments similar to those used in arriving at the upper bound (|2U|) for the 
sum rate, we can use a jointly Gaussian description scheme with covariance matrix of 
wz's (K w ) taking the form (J23j) . any tuple (Ri, . . . , Rl) satisfying 



(Ri, . . . , Rl) ■ 





K x +K z 




Cov[x|ui, leS]+K z 




Cov[x| Ui , les] 


n 


Cov[x| Ui ]+K 3 





is achievable. Thus if we can find a K„ of the form in 
distortion constraints are met with equality, i.e., 

B s = Cov[x|u,, I G S] = [K- 1 + (!,..., I)K~J(I, . . . , I) 



\/SC {1, L} ( ilh) 
such that all of the 2 L - 1 



ti-i 



V5C{1, L}, 



(98) 
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where K^ s is the covariance matrix for all K Wf , I e S, then the achievable region matches 
the outer bound and we would have characterized the rate region of the multiple descrip- 
tion problem. 

From the above discussion, we see that for some choice of distortion constraints of 
the multiple description problem, we can indeed do this: First choose L + l distortions 
(Di, D 2 , D L , D ) such that they satisfy the condition for Theorem El for the 

multiple description problem with individual and central receivers. Next we can solve 
for the K. w which is the covariance matrix of (w 1; . . . , w^) for the sum-rate-achieving 
jointly Gaussian description scheme. For any other SC{1, . . . , L}, this scheme results 
in distortion D5 = [K^ 1 + (I, . . . , I)K~*(I, . . . , I)*] -1 . Finally we choose these D^'s as 
the other distortion constraints. Now we have a general multiple description problem with 
2 L — 1 distortion constraints D$ for each S C {1, . . . , L}, and hence we can find a K w 
of form (J23j) such that all of the 2 L — 1 distortion constraints are met with equality. Thus 
(JSfiJl is actually the rate region and it can be achieved by a jointly Gaussian description 
scheme. 



Appendix 

A Useful Matrix Lemmas 

In this appendix we provide some useful results in matrix analysis that are extensively 
used in this paper. 

Lemma 5 (Matrix Inversion Lemma). [25, Theorem 2.5] Let A be an m x m non- 
singular matrix and B be an n x n nonsingular matrix and let C and D be m x n and 
n x m matrices, respectively. If the matrix A + CBD is nonsingular, then 



(A + CBD) 



1 



A 



A" 1 C(B- 1 + DA^C^DA 



Lemma 6. [25, Theorem 2.3] Suppose that the partitioned matrix 




is invertible and that the inverse is conformally partitioned as 
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If A is a nonsingular principal sub-matrix o/M, then 



X =A^ + A X B(D - CA- 1 B)- 1 CA 

Y = - A X B(D - CA^B) -1 , 
U = - (D - CA^B^CA -1 , 

V =(D - CA^B) -1 . 



(99) 



Lemma 7. [25, Theorem 6.13] Let E G M n be a positive definite matrix and let F be an 
n x m matrix. Then for any m x m positive definite matrix G, 



Lemma 8. [25, Theorem 6.8 and 6.9] Let A and B be positive definite matrices such 
that A y B (A B). ITien, 



B Proof of Lemma [T] 

Define an i.i.d. random process {z[m]}, m — 1, . . . , n of A/"(0,K 2 ) Gaussian random 
vectors, where z[m], m = 1, . . . , n are independent of x n and Cj, I — 1, . . . , L. Form a 
random process y n = (y[l], . . . , y[n])* by 

y[m] = x[m] + z[m], m = 1, . . . , n. 




(100) 



|A|HB| (|A|^|B|), 
A^B" 1 (A^B^ 1 ), 

A l/2^ B l/2 



(101) 
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It follows that {y[?ri]} is an i.i.d. random process of Af(0, K. y ) Gaussian random vectors, 
where K y = + K z . Then 

I(C i; C 2 ; C L ) + I{C U C L ;x") 

L 

= ^H(C l )-H(C 1 ,-.. ,C L )+I(C 1 , C L ;x n ) 
1=1 

L 

>Y,H(Q)-H(C lr .. ,C L )+I(C 1 , C L ;x") 
1=1 

L 

- (^F(a|y n ) C L |y")) 
i=i 

L 

= E W) " h (y n \Ci)) - h(y n ) + h(y n \C u . . . , C L ) + h(x n ) - h^C,, . . . , C L ) 
i=i 

L 

=h(x n ) + (L- l)h(y n ) - £ Ky n \Q) + Ky n \C u ...,C L )- h(x n \C u ...,C L ). 

1=1 

(102) 

Since x n and y n are Gaussian vectors, for the first two terms in (jl(J2|) . we have 
h(-x. n )=\\og(2ne) Nn \K x \ n , 

(103) 

h(y n ) =ilog(27re) 7Vn |K y r = ~ log(2vre)^|K x + K 2 | n . 
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We also have the following bound on h(y n \C{) for 1 = 1, . . . , L: 

n 

h( y n \Q)<Y,KyH\Ci) 

m=l 

<^^og(2^e) Ar |Cov[y[m]|Q]| 

1 - 

-^Cov[y[m]|q] 



m=l 



! 71 

<- \og(2>Ke) Nn + -log 



1 71 

~log(27re)"» + ~]o 6 



~log(27re)^ + ^lo£ 



m=l 
n 



-VCov[(x[m] +z[m})\Q} 

m=l 

n 

- VCov[x[m] 1(7,1 +K 



m=l 



<ilog(27re) Af " + |log|D, + K z | 

= ilog(27re)^ ft |D { + KX- 

Next we bound the last two terms of (jl(J2j) as follows. 

h{y n \C u C^-^ld, C L ) 

= h(y n \C u C L )-Mx"|z", d, 
= h(y n \C 1 , C £ )-/i(y>", C l5 
= /(y^ld, C L ). 



C L ) 
Cl) 



Letting 



K c [m] = f Cov[x[m] — x [m]], 



(104) 



(105) 



(106) 
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we have 



i{y n -X\c u c L ) = h( z n \c u c L )-Mz n |y n ,^i, c L ) 

= h(z n )-h(z n \y n -±lC 1 , C L ) 

> h(z n ) - h{z n \y n -x") 

n 

= {h(*H) - h(m[m]\m[l], . . . , z[m - 1], y» - *»)) 

m=l 
n 

> (/i(z[m]) — /i(z[m]|y[m] — xo[m])) 



m=l 
n 



(107) 

J(z[m]; x[m] — x [m] + z[m]) 

m=l 
- 1 

EN 



(a) 
> 

2 

m=l 

(*)n. |D + K 2 
> — lor 



|K c [m] 4 


-K z [m]| 


|K C | 


m]| 



|Do| ' 

where (a) is from (jlU6j) and [26, Lemma II. 2] . The justfication for (b) is from the convexity 
of log in A and Q. From (fTU5J) and (fTU7J) we have 

h(y»\C lt ...,C L )- h(*»\C u . . . , C L ) > £ log |D, ;J" Kz| . (108) 



Combining (fTU2l. (fTU3l and (fTUKj) . we have 



IK^IK^ + K.I^IDo + K, 



/(d; C 2 ; Ci) + /((?!, C L ; x") > - log ' ^ (1Q9) 

|D |n|D/ + K 2 | 

Taking the supremum over all positive definite K 2 , we can sharpen the lower bound in 
(USD: 



2^ J(Ci; C 2 ; . . . ; C L ) + /(Ci, . . . , C L ; x ) > sup^ - log . 

(110) 



iDoini^ + K, 

i=i 
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C Proof of Proposition U 



Conditioned on y, the collection of random variables (u 1; . . . , u^) are jointly Gaussian 
and thus we have 

1 n|Cov[u,|y]| 

V/i(ui|y) -ft(u 1} u L |y) = - logy— ^ —r. (Ill) 

^ 2 |Cov[ui, u L |yJ| 

From MMSE of from y we have 

Cov[u ; |y] = K x . + K Wl - K X (K X + K^K^ I = 1, . . . , L (112) 

and 

Cov(u 1; . . . , u L |y) = J ® K x + K w - J ® (K^K, + K,)- 1 ^) , (113) 
where J is an L x L matrix of all ones and ® is the Kronecker Product. 

By Fischer inequality (the block matrix version of Hadamard inequality, see [25, The- 

L 

orem 6.10]) we know that J~[ |Cov[u/|y]| = |Cov[ui, . . . , Ux,|y]| if and only if the off- 

i=i 

diagonal block matrices of Cov[ux, . . . , Ujjy] are all zero matrices. Thus we have 

L 

^2 H u i\y) - H u u ■ ■ ■ , u L |y) = o 

i=i 

if and only if 

K X -A = K Z (K :E + K 2 )- 1 K X , (114) 
or equivalently, if and only if 

K 2 = K X (K X - A)" 1 !^ - K x . (115) 
To get a valid K z y 0, we need the additional condition -< A -< K x . 

D Proof of Lemma [2] 

First we assume A y 0, and hence 

[A^+ai ... i)k^(h ... i/]- 1 

= A — A (I I ... I) [K w + (I I . . . I)* A (I I ... I)]" 1 (I I ... I)* A 



A- A (I I ...I 

L 



diag{K Wl + A, K W2 + A, ... K WL + A} 
A-A^[K W! +A]- X A. 



-i 



(I I... I)* A (H6) 



i=i 
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Thus, 



(II... I)K~ (I I ... I 



A-A^K^ + A^A 

^(K^+A)- 1 ) 
1=1 J 



--A' 1 - A X A 



-T -1 



AA X A 



AA" 1 - A" 1 



(117) 



^(K^+A)- 1 -A 



. i=i 



When A is singular, we can choose 5 > such that A + el y for e G (0, 5), and thus 
we can apply the previous argument and let e — ► + in the end. 



E Proof of Lemma El 



We use induction. First consider the matrix 



We have 



K W1 A 
—A K„, 2 



A 2 y ^K W2 y AK^A 



4=> K W2 + A y AK^jA + A 

«=► (K W2 + A)- 1 -< (AK;;A + A)- 1 

^ (K W2 + A)- 1 -< A- 1 - (K W1 + A)" 1 

(K W1 + A)- 1 + (K^ + A)- 1 -< A' 1 
l (118) 
<= ^(K Wi +A)- 1 ^A- 1 

^4(K W0 + A)- 1 ^A- 1 
K W0 + A >- A 
K wo >- 0, 



where (a) is from (|27|) . 
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Next we define 



/K m -A -A 



A Kj/ir, A 



A 



A K M , fc _ 1 A 



-A . . 

\-A ... -A -A K W J 

and suppose >- for k = 3, . . . , I — 1. Then 

A, 5- ^ K Wi >- A(I, I, . . . , I)Ar_\(I, I, • • • , I)*A 



K Wl y A 



^(K^+A)-M -A 



,it=i 



A 



' i-i 



^(K^ + A)- 1 -A 



K^+A)- 1 ^ 



A 



A + A 



' i-i 



^(K^ + A)- 1 -A 



A + A 



K Wl + A)' 1 -< A- ] 



' i-x 



-i 



-i -i 



^(K^ + A)- 1 -A + A 



,fc=i 



i-i 



(K 1 , ; +A)- 1 ^A- 1 -^(K ?Ufc +A)- 1 

k=l 

L 

^(K^ + A)- 1 ^- 1 



k=l 



(K W0 + A)- 1 ^A- 1 
K W0 + Ay A 
K wo y 0, 



where (b) is from (J27J). 



F Proof of Proposition El 

First we prove that 

Dq 1 + K; 1 - D^ 1 - D 2 1 y A* y 0. 
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Proof. We have 



A* 



Thus 



fK,. 



K^q) 2 (K Wl K^J 2 (K^ K W0 )(K U , 1 K wo ) 



fK 



'"1 



-K^-K 



(120) 



A* y 



(K 



K wo )i 



'K W1 - K 



(K m - K 

(K m - K. 



■W ) 2 

1 



■W ) 



-a(K 



■u.>2 



K W0 )(K 



■U.'l 



K 



1 

2 



(K 



U>1 



■K W0 )5 



>- K 



U'O 



K W0 )(K 



K 



>- (K mi -K W0 ) 2 K 



■Wq\^-W1 K^q) 



K W0 )(K„ 



5- (K W1 - K 



'-tUQ (K; 



K W2 - K 



K 
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•«'2 

-„, j~ 2 K 
-u>o Kuiq (K^ K 

K mo >- K 



-tt'i 



(-I+^-K^,)" 1 )^ 



K W2 - K 



-M)0 



^-WQ ^-WQ (K. 

■ K W2 y K W0 K-i(K^ - K; 1 )" 1 ^ 

• k W2 >- (k-; - K-i)- 1 

• K-J + K~l -< K w l 

Do 1 + K; 1 - Dr 1 - D 2 1 y 0. 



K U , )~ 1 K, 



til J 



(121) 
□ 



The proof of 



D + K a 



Di 



D 2 y =>• A* -< K a . 



is similar and hence is omitted. 



G Proof of Lemma 31 



-1 



-1 



= [(K wo + A*)- 1 ^ + (K wo + A*)A h 

( = } (I + K^A^-^K^ + A*) 

= (I + K^Ai)" 1 ^ + A* - (I + K tt0 A 1 )A* 

® (I + K^AO-^+A* 
= (K-i(I + K UJ0 A 1 )) _1 + A* 

= (K- f ; + A 1 ) _1 + A*, 



A* 



(122) 
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where (a) and (b) are from AiA* = 0. 
IDS + KJ 



ID* I 
l-^ol 



|I + DS _1 K 



|I + (Do 1 + A 1 )K«| 
II + D^ + AiK*! 

|I + Do 1 K, + A 1 ((I -A*)- 1 -I) | (123) 

|I + B^K Z + Ax(I - A*) ((I - A*)- 1 - I) | 
II + D^K*! 
|D + K 2 | 



I Dp 



where (c) is from AiA* = 0. 



H Proof of Equations (CO) and (ED) 

We first prove the following lemma. 

Lemma 9. Let D be an N x N matrix such that -< D -< I. Let K = (D _1 — I) -1 . 
Choose e > such that K — el y 0. Define 

D(6)^ / [(K-eI)- 1 +l]- 1 . 
T/ien, t/iere exzst constants b\ > b 2 > 0, such that 

D - b x el + o(e) -< D(e) -<T>- b 2 el + o(e) 

Proof. There exists anJVxiV orthogonal matrix Q such that 

QKQ* = diag{fci, . . . , k N }, 
where fcj > are eigenvalues of K. We have 

QDQ* = Q(K 1 + I) _1 Q* 
= diag 



ki k 



N 



1 + ki 1 + k 



N 
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and 



QD(e)Q i = Q 



fK-ell^ + I 



n -1 



(diag{fci, k N } — eiy 1 + I 
k\ — e k^ — e 



diag 
diag 



1 + fci — e 1 + fcjv — e 

+ o(e), . 



k 



N 



l + k N + 



1 + fci (1 + fci) 2 
We now have 

QDQ* - bxel + o(e) -< QD(e)Q* -< QDQ* - 6 2 el + o(e 

where &i > &2 > are some constants. Hence 

D - 6id + o(e) -< D(e) -< D - £> 2 el + o(e). 



+ 



Equations (fTOj) and (fTTj) are a direct consequence of this lemma. 

I Proof of Equation (E3) 

We first prove the following lemma. 

Lemma 10. Let A be an N x N matrix such that ^ A -< I. Let K 2 = (I 
Choose e > suc/i A + el -< I. Define 

^^[I-CA + el)]- 1 -! 

T/ien, i/iere exist constants c\ > c 2 > sitc/i 

K 2 - ciel + o(e) -< K t {e) -< K z - c 2 el + o(e). 

Proof. There exists anJVxiV orthogonal matrix Q such that 

QAQ* = diag{ai, . . . , a a?} 
where a« > are the eigenvalues of A. We have 

QK,Q* = Q((I- A)- 1 -I)Q t 

j . [ fli a N 1 
= diag <^ , . . . , V , 
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and 

QK,(e)Q* = Q((I - (A + el))- 1 - I)Q* 

ax + e a N + e 

diag 



1 — CL\ — e 1 — ajv — e 

ai (2oi — l)e ^ . . oat (Zcin — l)e 

1 — ai (1 — ai) 2 ' 1 — oat (1 — on) 2 

We now have 

QK 2 Q* - cid + o(e) -« QK 2 (e)Q* H QK 2 Q* - c 2 el + o(e), 
where ci > c 2 > are some constants. Hence 

K 2 - Cl el + o(e) -< K 2 (e) -< K 2 - c 2 el + o(e). 

□ 

Equation ()72)1 is a direct result of this lemma. 



J Proof of equation ( IHzl ) 



We first prove the following lemma. 

Lemma 11. Lei A be an N x N matrix such that -< A =<! I. Choose e > snc/j 
A - el >- 0. Define 

K z (e) d = f [I - (A - el)}' 1 - I. 
Then, for any E and F suc/i that -< E ^ I and -< F I 7 we have 

lim |g + gf>| =l. 
<-o |F + K,(e)| 

Proof. There exists anJVxiV orthogonal matrix Q such that 

QAQ* = diag{ai, . . . , a N }, 

where < Oj < 1 are eigenvalues of A. Without loss of generality, we suppose a x = 
1, . . . , a p — 1, a p+1 < 1, . . . , a?v < 1. 

We have 

QK 2 (e)Q* = Q((I - (A - el))" 1 - I)Q* 

1 — e 1 — e a p+ i — e — e 



diag 



e el- a p+ i + e 1 — ajv + e 
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and since 

|I + K z (e)| > |E + K 2 (e)| > \K z (e) 



\K z {e)\ ~ |F + K z (e)| " |I + K z (e)|' 
we have 

Hm^f^!. 
e-o |F + K 2 (e)| 

□ 

Equation (|82|) is a direct consequence of this lemma. 



K Proof of Equation 

We would like to have a property similar to as e\ approaches zero, and a property 
similar to (J82j) . as e 2 approaches zero. To see this is the case, we need the following 
lemma. 

Lemma 12. 

AiK,(ei =0,e 2 ) = 



Proof. Since 
and 



QAiQ'QA*Q* = 
QA*Q* = diag(0, . . . , 0, 1, . . . , 1, a p+q+1 , a s 



V Q 



QA*Q - e 2 E 2 = diag(0, . . . , 0, 1 - e 2 , . . . , 1 - e 2 , a p+q+1 , . . . , a s ), 



we have that 



QA*Q i (QA*Q i - e 2 E 2 ) = 0. 



Thus 



QA x K*(ei = 0, e 2 )Q* = QAiQ*Q ((I - A* + Q*e 2 E 2 Q) _1 - I) Q* 

= QAiQ* ((I - QA*Q* + e.E,)- 1 - I) 
= QAiQ^I - QA*Q* + e 2 E 2 ) ((I - QA*Q* + e.E,)- 1 - I) 
= 0. 



□ 
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Using this lemma, we can show a property similar to (J55|) as ei approaches zero. First 
note that similar to case 2, we have 

D^ 1 + Ax - e 2 e 2 I + o(e 2 ) -« D„ ^ei, e 2 ) H D 1 + A 1 + e 1 e 1 I + o{e x ) 

where e\ > and e 2 > are constants. Hence we have 

|D (e 1 = 0,e 2 )+K 2 (e 1 = 0,e 2 )| |T , __j 



|D (ei = 0,e 2 ) 



= |I + Do A (ei = 0,e 2 )K z (e 1 = 0,e 2 )| 

> |I + (Do 1 + Ai — e 2 e 2 I)K z ( ei = 0, e 2 )\ 
= II + DoXCei =0,e 2 ) - e 2 e 2 K z (ei =0,e 2 )| 
|D + K 2 ( £l = 0, eg) - e 2 e 2 D K 2 (ei = 0, e 2 ) 
iDnl 



Similarly, we have 



|D (ei = 0, e 2 ) + K*(d = 0, e 2 )| < |D + K,(ei = 0, e 2 ) 



Thus 



p (e 1 = 0,e 2 ) - p c 



r r 1, |I + K 2 (ei, e 2 )|( L - 1 )|D (e 1 , 6 2 ) + K 2 ( £l , e 2 ) 
hm hm - log 

|D (e 1 ,e 2 )|n|DKei 1 e 2 )+K z ( ei ,e 2 )| 
i=i 



]im 1 lQg II + K^^O^^I^IDo + K^^O.eOI (124) 
<J ~" 2 |D | EI |Dj(d = 0, e 2 ) + K,(ei = 0,e 2 )| 



X i 1 



2 b |D |' 
where the last step is similar to (j82j) . 
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